Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation

This paper aims to show that robots equipped with a vision-based tactile sensor can perform dynamic manipulation tasks without prior knowledge of all the physical attributes of the objects to be manipulated. For this purpose, a robotic system is presented that is able to swing up poles of different masses, radii and lengths, to an angle of 180 degrees, while relying solely on the feedback provided by the tactile sensor. This is achieved by developing a novel simulator that accurately models the interaction of a pole with the soft sensor. A feedback policy that is conditioned on a sensory observation history, and which has no prior knowledge of the physical features of the pole, is then learned in the aforementioned simulation. When evaluated on the physical system, the policy is able to swing up a wide range of poles that differ significantly in their physical attributes without further adaptation. To the authors' knowledge, this is the first work where a feedback policy from high-dimensional tactile observations is used to control the swing-up manipulation of poles in closed-loop.Comment: Accompanying video: https://youtu.be/4rG-o2Cz3-

Towards vision-based robotic skins: a data-driven, multi-camera tactile sensor

This paper describes the design of a multi-camera optical tactile sensor that provides information about the contact force distribution applied to its soft surface. This information is contained in the motion of spherical particles spread within the surface, which deforms when subject to force. The small embedded cameras capture images of the different particle patterns that are then mapped to the three-dimensional contact force distribution through a machine learning architecture. The design proposed in this paper exhibits a larger contact surface and a thinner structure than most of the existing camera-based tactile sensors, without the use of additional reflecting components such as mirrors. A modular implementation of the learning architecture is discussed that facilitates the scalability to larger surfaces such as robotic skins.Comment: Accompanying video: https://youtu.be/lbavqAlKl9

Chain of Hindsight Aligns Language Models with Feedback

Learning from human preferences is important for language models to be helpful and useful for humans, and to align with human and social values. Prior work have achieved remarkable successes by learning from human feedback to understand and follow instructions. Nonetheless, these methods are either founded on hand-picked model generations that are favored by human annotators, rendering them ineffective in terms of data utilization and challenging to apply in general, or they depend on reward functions and reinforcement learning, which are prone to imperfect reward function and extremely challenging to optimize. In this work, we propose a novel technique, Chain of Hindsight, that is easy to optimize and can learn from any form of feedback, regardless of its polarity. Our idea is inspired by how humans learn from extensive feedback presented in the form of languages. We convert all types of feedback into sentences, which are then used to fine-tune the model, allowing us to take advantage of the language comprehension capabilities of language models. We condition the model on a sequence of model generations paired with feedback. By doing so, models are trained to generate outputs based on feedback, and models can learn to identify and correct negative attributes or errors. Applying our method to large language models, we observed that Chain of Hindsight significantly surpasses previous methods in aligning language models with human preferences. We observed significant improvements on summarization and dialogue tasks and our approach is markedly preferred in human evaluations.Comment: included new result

MMP-2, MMP-9 and activin A blood levels in patients with breast cancer or prostate cancer metastatic to the bone.

Background: The clinical significance of the circulating levels of activin A and matrix metalloproteinase-2 (MMP-2) and -9 (MMP-9) was investigated in patients with breast cancer (BC) or prostate cancer (PC) with (M1) or without (M0) bone metastasis. Patients and Methods: MMP-2, MMP-9 and activin A blood concentrations were measured by enzyme immunoassays in 79 cancer patients and in 57 healthy blood donors (HS) who served as a control group. The diagnostic accuracy of these molecules to discriminate between M0 and M1 patients was evaluated by the receiver operating characteristic curve (ROC) and compared to that of tumor markers CA15.3 or prostate-specific antigen (PSA). Results: Activin A and MMP-2 were significantly increased in BC and PC patients as compared to sex-matched HS while MMP-9 levels were more elevated only in the PC patients. Interestingly, in the PC patients, activin A levels were significantly higher than those measured in the BC patients. In this latter group, activin A and CA15.3 but not MMP-2 or MMP-9 were increased in the M1 patients as compared to M0 patients. Furthermore, a significant relationship was also highlighted between activin A concentration and the number of bone metastases and tumor grade, between MMP-9 and tumor grade, and between MMP-2 and CA15.3. ROC curve analysis showed a good diagnostic accuracy for activin A and CA15.3 but a poor accuracy for MMP-2 and MMP-9 in discriminating between M0 and M1 patients. However, CA15.3 retained the best diagnostic accuracy in this respect. In the PC group, only activin A and PSA levels were significantly increased in the M1 patients as compared to the M0 patients. A similar although not statistically significant trend was noted for MMP-9. Interestingly, a significant correlation was observed between PSA and activin A and MMP-9, and between Activin A and Gleason score and the number of skeletal metastases. ROC curve analysis showed a good diagnostic accuracy for activin A, MMP-9 and PSA and a poor diagnostic accuracy for MMP-2 in detecting M1 patients. However, PSA showed the highest diagnostic accuracy. Conclusion: Activin A, MMP-2 and MMP-9 may be regarded as possible therapeutic targets in the treatment of metastatic bone disease. However, their usefulness as additional markers of bone metastasis remains to be better define

A general framework for high-resolution robotic tactile sensing: design, simulation, and learning

In order to fulfill their potential in the manufacturing and retail sectors of the modern world, autonomous machines need to be able to perceive and react to contact with their surroundings, both to enhance their capabilities, as well as to increase operational safety. This thesis investigates solutions to the contact sensing problem of robotic systems, pivoting on the development of a vision-based tactile sensing principle that provides rich information upon physical interaction with the environment. The sensors based on such a principle are low-cost, scalable to large surfaces and straightforward to manufacture. However, they do not directly measure physical quantities, but rather provide raw data in the form of what are generally known as tactile images. In this work, a machine learning-based data processing framework is presented to address three main requirements, namely, sensing accuracy, efficiency, and generalization across tasks and contact conditions. State-of-the-art sensing accuracy, at a spatial resolution comparable to that of the human fingertip, is achieved through a deep neural network that maps the raw tactile images to the three-dimensional force distribution applied to the sensing surface, which provides a compact and generic representation of the contact state. In fact, the force distribution contains information about the location and the intensity of shear and pressure forces, as well as about the shape and the number of the possibly distinct contact regions. In addition, it provides an interpretable physical quantity that is shown to be very practical for planning higher-level robotic tasks. The size of the neural network is kept compact to ensure real-time inference. However, in the context of data-driven methods, efficiency is also a concern with regard to training data requirements. In this thesis, accurate finite element-based simulations enable the synthetic generation of raw tactile data under a variety of contact conditions. The same simulations also yield appropriate force distribution labels, which are otherwise not possible to collect with currently existing commercial force sensors. Hence, the deep neural network is entirely trained with synthetic data, avoiding the need for real-world data collection. A strategy is then presented that facilitates a seamless transfer of the inference model from simulation to reality, retaining high sensing accuracy. In addition, the model transfers across sensors of the same type without further training. The simulation training facilitates data collection across different scenarios, such as the contact with arbitrarily shaped objects or the combination of shear and pressure interactions. An appropriate choice of learning architecture shows generalization capabilities when applied to contact conditions not present in the training dataset. Beyond the pure sensing task, a proof-of-concept robotic system is presented that fully leverages the versatility of the tactile sensor. The system achieves dynamic manipulation of objects with unknown physical properties solely through the use of tactile feedback fed to a closed-loop control policy trained with a deep reinforcement learning algorithm. In a separate part, this thesis discusses a different research topic, where past experience data are employed to improve the trajectory tracking performance of autonomous systems. This is achieved by estimating unmodeled disturbances over different trials, and including them in the formulation of a computationally efficient model predictive control framework. The approach is demonstrated on two flying vehicle applications, namely, on a vehicle powered with electric ducted fans and controlled through thrust vectoring, and on a quadcopter that aims to balance a pendulum rod during flight

Sim-to-Real for High-Resolution Optical Tactile Sensing: From Images to Three-Dimensional Contact Force Distributions

The images captured by vision-based tactile sensors carry information about high-resolution tactile fields, such as the distribution of the contact forces applied to their soft sensing surface. However, extracting the information encoded in the images is challenging and often addressed with learning-based approaches, which generally require a large amount of training data. This article proposes a strategy to generate tactile images in simulation for a vision-based tactile sensor based on an internal camera that tracks the motion of spherical particles within a soft material. The deformation of the material is simulated in a finite element environment under a diverse set of contact conditions, and spherical particles are projected to a simulated image. Features extracted from the images are mapped to the three-dimensional contact force distribution, with the ground truth also obtained using finite-element simulations, with an artificial neural network that is therefore entirely trained on synthetic data avoiding the need for real-world data collection. The resulting model exhibits high accuracy when evaluated on real-world tactile images, is transferable across multiple tactile sensors without further training, and is suitable for efficient real-time inference.ISSN:2169-5172ISSN:2169-518